Skip to content

ctrl#1533 phase 2: swap Value map/list host carriers to im-rc HAMT/RRB (persistent updates, copied-counters collapse to 0) - #4672

Merged
briansrls merged 25 commits into
mainfrom
session/neat-ibex-297
Jun 11, 2026
Merged

briansrls merged 25 commits into
mainfrom
session/neat-ibex-297

Conversation

@briansrls

@briansrls briansrls commented Jun 11, 2026 •

Copy link
Copy Markdown
Contributor

Phase 2 of the persistent-structures plan (ctrl #1533; phases 0/1 = #4653, #4660, #4666): swap the v2 interpreter's host carriers behind the Value surface to the carriers phase 1 declared. No DSL-observable semantic change.

What changed

  • Value::Map now carries Rc<im_rc::HashMap> (HAMT — matches map_carrier ^hash_array_mapped_trie), Value::List carries Rc<im_rc::Vector> (RRB — matches list_carrier ^relaxed_radix_balanced_tree, O(log n) concat). im-rc over rpds because it provides BOTH declared carriers (rpds::Vector has no concatenation) and is Rc-backed, matching the interpreter's Rc discipline (rationale comment in Cargo.toml).
  • The Rc wrapper stays: value_rc_identity (pure-call memo keys) and the retained-accounting sharing dedup key off Rc::as_ptr. M-C law 1 stays one-way — pinned by a host-side probe (two runs build pointer-distinct, ==-equal values).
  • map_insert/merge and list_push/concat are now persistent updates; the *_entries_copied / *_items_copied counters count nothing there anymore. Only the FreeMonoid flatten chokepoint (ctrl#1476 B1) still materializes items — exactly the L1/L2 split ctrl chore(r3-debt-paydown): velocity-tripwire script + first 2026-05-02 measurement (FIRED 25.33:1) #1534 documents.
  • B1 alias transparency preserved: native List and Empty/Cons chains route through one value_to_list_carrier bridge (no per-representation operand arms; list_free_monoid_chokepoint_test still green).

Receipt (the phase-2 number)

GUNBC_INTERP_STATS=1 claim_batch over the 9 value_carrier_laws witnesses:

map_insert    18 calls    0 entries copied   (phase 0: copied ≈ n(n−1)/2, triangular)
list_concat    6 calls    0 entries copied
fm_flatten    49 calls   49 items materialized   (the remaining, intended cost)

interp_stats_test re-pins the new semantics: the workloads that copied 6/9/3 entries on main now copy 0 (tests renamed to say so).

New witnesses

  • value_carrier_laws.dag: list/map prior-version persistence (Driscoll et al. — derived version leaves the base valid; the failure class a carrier swap can introduce is in-place mutation through a shared handle). 11 claims in the file, all PASS.
  • value_carrier_swap_test.rs: host-side at-scale probes (n=500, past im-rc's 64-element inline chunks so HAMT/RRB tree paths actually run): insert-order-independent equality, overwrite path-independence, prior-version validity, identity non-observability.

Gates run

  • cargo test -p v2-compiler-tests: 27→24 failures vs baseline — all 24 are the pre-existing pipeline::* failures (verified identical set on the unmodified branch); the 4 new probes pass.
  • stage0 freshness: regen_stage0 --verify → "committed stage0 matches fresh self-compile".
  • parity (release, --exact): PASS.
  • cargo clippy --all-targets -- -D warnings, cargo fmt --all --check: clean.
  • 9/9 carrier-law witnesses PASS by execution (claim_batch --claim-run).

Phase 3 (interning + positional records, M-D) and phase 4 (re-measure) follow separately.

🤖 Generated with Claude Code

Brian Searls and others added 24 commits June 10, 2026 23:35
Context-scoped MutationCounters on the copy-on-update primitives
(map_insert/merge/list_push/concat/set ops: calls + entries copied — the
triangular/quadratic receipt), thread-local flatten counters on the
free_monoid_to_vec chokepoint (fires inside Value::eq, so no ctx; two
fixed-size integers, not a cache), and a sharing-aware retained-value
byte accounting walk (per-variant counts/bytes, visited-set dedup) on
InterpContext. claim_batch prints the report under GUNBC_INTERP_STATS=1.

Receipt on the real v4 gate workload (7 affected_testgen witnesses, all
PASS): native mutation primitives 0 calls — v4 collections are closure
chains + FreeMonoid trees, so the host copy cost rides the flatten
chokepoint, confirming ctrl PR #1534's layer analysis. Retained: 21.8 MB
/ 219K allocations (Record 12.7MB with 48K sharing hits; 31K un-interned
Strings) — the M-D interning/positional-record receipt baseline.

Read-only tooling per ctrl#1533 phase 0; no semantic change to any
evaluation path (counters + opt-in report only).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…latten in claim_batch

Review #28288 (cursor/composer-2.5) on #4653: the copy-work counters mixed
definitions across dispatch paths, breaking the single-receipt property (P2).

- One definition per counter, by operation semantics, not dispatch path:
  add-one ops (map_insert/list_push/set_insert) count the receiver's
  pre-existing entries; merge ops (map_merge/list_concat/set_union) count
  both operands' entries. Method .concat/.append/.push buckets by what the
  arg IS (collection -> concat, atomic -> push); binop + and builtins agree.
- set_union gets its own row (was folded into set_insert — same P2 class).
- builtin map_merge instrumented (was counted on the method path only).
- claim_batch fm_flatten row is now a delta sampled across the witness loop,
  matching the context-scoped counters next to it.

Tests pin the unified semantics: concat counts both operands (9 for the
3⊕1 then 4⊕1 chain), atomic append lands in list_push with receiver-only
copy-work, and neither bucket leaks into the other.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…arrier_laws; strengthen insert-order witness to 3 keys

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…iveContract + M-D name-interning declarations

M-B: PrimitiveContract gains carrier_cost: CarrierCostSensitivity. 12
carrier-sensitive primitives declare both arms (ephemeral = copy-before-
update, what the v2 interpreter does today; persistent = the declared M-A
carrier cost); 49 carrier-insensitive primitives marked explicitly.

M-D: name-interning facts at the v4.std.value_carrier authority — type/
variant/field names are references into the resolved graph's declared name
set; positional-record layout derives from this fact (phase 3 implements).

New witness name_interning_covers_all_domains_holds (8/8 green).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Value::Map now carries Rc<im_rc::HashMap> (HAMT, matches map_carrier
^hash_array_mapped_trie) and Value::List carries Rc<im_rc::Vector> (RRB,
matches list_carrier ^relaxed_radix_balanced_tree). Rc wrapper kept:
value_rc_identity memo keys and accounting dedup rely on Rc::as_ptr.

map insert/merge and list push/concat no longer copy entries; the
*_entries_copied / *_items_copied counters now count only FreeMonoid
chain flattens — the copied-counter collapse is the phase-2 receipt.
List ops route native List and Empty/Cons chains through one
value_to_list_carrier bridge (B1 alias transparency preserved).

New witnesses: list/map prior-version persistence in
value_carrier_laws.dag; host-side at-scale probes (n=500, past inline
chunks) in value_carrier_swap_test.rs.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…s behind Value

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

# Conflicts:
#	src/v4/test/claim/std_grounding/value_carrier_laws.dag
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@briansrls briansrls changed the title memory usage ctrl#1533 phase 2: swap Value map/list host carriers to im-rc HAMT/RRB (persistent updates, copied-counters collapse to 0) Jun 11, 2026
@briansrls
briansrls marked this pull request as ready for review June 11, 2026 14:06
@briansrls

Copy link
Copy Markdown
Contributor Author

Re claude/composer review nit (redundant inner parens in ~16 list_value((x)) sites, e.g. v2_interpreter.rs:1183,1359,2378): confirmed real — leftover from a scripted edit during the carrier swap; cosmetic only, no behavioral effect (and the reviewer marked it non-blocking).

Deliberately NOT pushing a cleanup commit to this PR: it's merge-ready on this HEAD (2 approvals, CI green) and a cosmetics-only push would stale both approvals while the operator is merging manually. The paren cleanup is queued as the first commit of the ctrl#1533 phase-3 PR, which touches the same file.

— sent from neat-ibex-297

@briansrls
briansrls merged commit f5d42be into main Jun 11, 2026
11 checks passed
@briansrls
briansrls deleted the session/neat-ibex-297 branch June 11, 2026 16:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant